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Abstract 

In this paper we study a class of fractional elliptic problems of the form 

J (-A) s u = f{x,u) in Q 

\ u = mR N \fl, 

where s € (0, 1). We prove nonexistence of positive solutions when f2 is star- 
shaped and / is supercritical. We also derive a nonexistence result for subcritical 
/ in some unbounded domains. The argument relies on the method of moving 
spheres applied to a reformulated problem using the Caffarelli-Silvestre exten- 
sion [11] of a solution of the above problem. The standard approach in the case 
s = 1 using Pohozaev type identities does not carry over to the case < s < 1 
due to the lack of boundary regularity of solutions. 

1 Introduction 

Let s € (0, 1) and N > 2s. In the present paper, we are concerned with the 
nonexistence of positive functions solving the fractional elliptic semilinear problem 



(1.1) 



(-A) s u = f(x,u) in SI, 

u = in l JV \a 
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in a domain Q C M. N . Problems of this type received immensely growing attention 
recently, while different versions of the nonlocal operator (— A) s related to Dirichlet 
boundary conditions are studied (see e.g. [5,8,12,15,29,32]). The version we consider 
in (1.1) is the one most commonly considered in analysis and probability theory. In 
probabilistic terms, it can be defined as the generator of the 2s-stable process in Q 
killed upon leaving Q. For our purposes, it is more convenient to give an analytic 
definition. We define (-A) s for any ip G C™(R N ) by 

<p{x) - <p(y) f <p(x) - <p(y) 



(i.2) ( -A)v(x) = p.v./ ^0l dy = ]i f 

J r n \x-y\ N+IS e -^J\x-y\ 



>£ \x - y\ N+2s 



dy 



for x G M. N , where P.V. stands for the principle value integral. We point out that 
this definition differs from the standard definition by a multiplicative constant. Via 
Fourier transform, (1.2) is equivalent to 



cjv,(-a)v(o = \t\ 2s m for e G m v 

p/ N+2s \ 

with the normalization constant Cn,s = s(l — s)it~ n ' 2 2 2s r ( 2 _ s ) ; see e -S- \Ji Remark 
3.11]. Thanks to Lemma 2.1 below, for any ip G C£°(R N ) we have the estimate 

(1.3) IWWI<g ^S forallzGR^, 

where C only depends on the support of ip. Let C l s denote the space of all measurable 
functions u : R N — > R such that J Rjv 1+ |j|A/+2 s dx < oo, and let 0, be an open set of 
R N . We define the Hilbert space V s * 2 {Q) as the completion of C£°(Q) with respect 
to the norm || • ||-p s ,2 induced by the scalar product (•, -)x>s,2 given by 



(1.4) {u,v) V s,2 = \ 

Jm 



(u(x) -u(y))(v(x) -v(y)) j 

R 2N \x - y\^+^ s 



We note that if £1 is a bounded Lipschitz domain, then V s ' 2 (Q) coincides with the 
Sobolev space {u G H S (R N ) : u = a.e. in R N \ fi}. We also observe that - for 
any u G V s ' 2 (Jl) - the Holder and the Hardy-Littlewood-Sobolev inequalities imply 
that 

! 1 ivn dx < C||^||2?-,2 for all u G V s ' 2 (fl) 

V l+\x\ N + 2s ~ " UD V ' 
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with a constant C > 0. In other words, V s ' 2 is continuously embedded in C\. As 
a consequence, by recalling (1.3) we may define (— A) s u for every u G V s ' 2 (ft) as a 
distribution by 

((-A) s u,p) := [ u{-A) s <pdx = {u,<p) vs ,2 for all tp G C c °°(^). 

In particular, given / G L / 1 oc (r2), we note that u G D S,2 (S1) solves the problem 
(— A) s u = f if and only if 

(1.5) (u,ip) vs ,2 = [ fipdx for all ip G C£°(f2). 

Throughout the paper, when we refer to solution of (1.1), we mean distributional 
solutions u G V S,2 (Q) in the sense of (1.5) with / = /(-,n(-)) G Lj oc (Cl). In order 
to state the main result of the present paper, we need to introduce a definition of a 
star domain which is slightly more general than usually considered in the literature. 
We say that an open set C R N is star-shaped (or a star domain) with respect to 
the origin G Q if for every x G SI we have tx G f2 for < t < 1. So in contrast 
to the standard definition, we also allow the star center to lie on the boundary of 
Q. This will be crucial in deriving results in unbounded domains. In particular, 
the punctured open unit ball -Bi(O) \ {0} is star-shaped with respect to the origin 
according to our definition. Our main result is the following. 

Theorem 1.1 Assume that Q is bounded and star-shaped with respect to the origin 
G f2. Suppose that f : ^\{0} x [0, oo) — > R is locally Lipschitz in its second variable 
uniformly in compact subsets of Q\ {0} and is supercritical in the sense that 

f the function A -> \-(N + 2s)/(N-2s) f{x -2/(N-2s) x ^ Xu) 
[ is non- decreasing on [1, oo) for every x£(! \ {0}, u > 0. 
Then (1.1) has no positive solution u G C{R N \ {0})nD s ' 2 (fi). 

We remark that for C 1 -nonlinearities / : f2 \ {0} xR4l the supercriticality 
assumption (1.6) is equivalent to 

(1.7) H f (x,u)>0 for all (x,u) G fi\ {0} x [0,oo), 
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where 

(1.8) Ht(x, u) := u-^- fix, u) — — - fix, u) x • V x fix, u). 

v ) )\ , ) du y ' N-2s y ' N-2s JK ' 1 

As a first consequence of Theorem 1.1 we have the following Pohozaev type result. 

Corollary 1.2 Assume that fl is bounded and star-shaped with respect to the origin, 
and let V E C 1 (Q\ {0}) satisfy 

sV(x) + ^ W(x) • x > for allxen\ {0}. 

Let u G V s ' 2 (ty n C(R N \ {0}), u > in R N be such that 
f {-A) s u + V(x)u = u p in Q, 

\ u = in R N \ ft 



(1.9) 



for some p > ^±|f . Then u = in R N . 

In the case where f2 is the unit ball in R N and V = 0, this gives an affirmative 
answer to a conjecture of Birkner, Lopez-Mimbela and Wakolbinger, see [3, p. 91]. 
We note that existence results for problem (1.9) in the subcritical range 1 < p < 
n~^.2s anc ^ f° r more general subcritical nonlinearities have been obtained recently by 
the first author in [17] and by Servadei and Valdinoci in [29]. 

In our next result the linear term is related to the relativistic Hardy inequality, 
see [18] and [17]. 

Corollary 1.3 Assume that £1 is bounded and star-shaped with respect to the origin 
and let u G V S > 2 (Q) n C(R N \ {0}), u > in R N be such that 

j (-A) s n - ~f\x\~ 2s u = u p inn, 
\ u = inR N \n 



(1.10) 



for some 7 Gl and p > ^±|f . Then u = in R N . 

Our next result is concerned with a singular nonlinearity. 

Corollary 1.4 Assume that Q is bounded and star-shaped with respect to the origin 
and let u G V S > 2 {VL) n C{R N \ {0}), u > in R N be such that 

( i-A) s u= \x\- a u p inn, 

(1.11) 

( u = in R N \ n 

for some a £R and p > max{l, ~ 2(J } . Then u = in R N . 
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This result should be seen in the context of the criticality of q = 2 $~<fJ = 
N ~iv-2s' 7 + 1 f° r t ne embedding of the Sobolev space V s ' 2 (fi) in the weighted space 
L q (Q; \x\~ a ). More precisely, if N > max(<7, 2s) and the underlying domain is 
bounded, V S ' 2 (Q) is continuously embedded in L q (tl; \x\~ a ) if and only if q < 2 ^~^ , 
and the embedding is compact iff q < 2 ^ 2 ^ , Note also that the existence of the 
embeddings in the subcritical range follows from the fact that 

V S >\R N ) ^ L 2(N-a)/(N-2s)^ R N. 

and this latter embedding can be seen as a version of the Stein- Weiss inequality [31]. 

Our next result is concerned with a class of unbounded domains. Slightly ex- 
tending a notion from [27], we say that an open set f2 is star-shaped with respect 
to infinity if there exists a point e G R N \ Q such that for every point x G Q. the 
half-line {e + t(x — e) : t > 1} is contained in Q. Up to suitable translation, it is 
equivalent to require and that \ fiis star-shaped with respect to in the 
sense defined earlier. 

Theorem 1.5 Assume that Q is star-shaped with respect to infinity. Let u G 
V s ' 2 (fl) n C(R N ) be nonnegative and such that 



(1.12) 



f (-A) s m = u p in Q, 

\ u = in R N \n 

for some 1 < p < -j^rff • Then u = in 



In fact, we will deduce Theorem 1.5 from Theorem 1.1 via a variant of the 
classical Kelvin transform, see Sections 2 and 3 below for details. 
Theorem 1.5 in particular applies to the cone-like domains Q T := {x G R N \ {0} : 
j^j > t\ for r G (—1,1). Here one may take e = —e^, where e^v is the TV-th 
coordinate vector, in the definition of star-shapedness at infinity. Since the half- 
space is a particular case with r = 0, we deduce the following corollary. 



Corollary 1.6 Let u G V S > 2 (R^) n C(R N ) be nonnegative and such that 

j (-A) s u = u p in R+, 

\ u = in R N \ R+ 



(1.13) 



for some l<p< Then u = in R N . 
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We remark that Theorem 1.5 does not apply to the case = R^. Indeed, in this 
case the critical problem with p = j^zff admits positive solutions which have been 
classified completely in [14]. Moreover, in the case Q = R N , s £ [1/2,1) and 1 < 
p < $rif > a nonexistence result has been obtained very recently and independently 
in [15] by de Pablo and Sanchez, see also [23] for s = 1/2 and 1 < p < $±|f . 

In order to explain our approach to obtain the nonexistence results, we need to 
compare (1.1) with the classical problem 



(1.14) 



—Au = f(x,u) in f], 

u = on dU. 



For (1.14), the analogue of Theorem 1.1 is true, and for strictly starshaped C 1 - 
domains O and C 1 -nonlinearities / on H x [0, oo) satisfying additionally /(-,0) = 
it can be derived from the Pohozaev type integral identity 

(1.15) / / Hf(x,t)dtdx + / / ul x-vda(x) = 0, 

Jo N — 2 J dn 

see e.g. [26, Theorem 5.2]. Here Hf is defined as in (1.8). Indeed, by (1.7) and the 
star-shapedness of 0, the LHS of (1.15) is nonnegative, and by unique continuation it 
is strictly positive if u ^ 0. The above integral identity can be derived by multiplying 
(1.14) with the functions u and x >— >■ x ■ Vn respectively and integrating by parts. 
The same strategy does not work for (1.1) since the problem is nonlocal and does 
not allow a simple integration by parts formula as in the case s = 1. More severely, 
in the case < s < 1 solutions of (1.1) are not of class C 1 up to the boundary 
even if the underlying domain is smooth. In particular, if x h-> f(x,u{x)) > is a 
nonnegative nontrivial function on 17, then any solution u of (1.1) fails to possess a 
finite normal derivative u v on dQ, see e.g. [3, Lemma 4.3]. 

The approach we follow here is inspired by Reichel and Zou [27] who used the 
technique of moving spheres to prove nonexistence results for cooperative elliptic 
systems. The moving sphere method can be seen as a variant of the method of 
moving hyperplanes (see e.g. [1,2,20,21,28]) and has been widely used to classify 
positive solutions of nonlinear elliptic problems, see e.g. [24] and the references 
therein. For the special case where the underlying domain is the entire space M. N , 
it has also been applied to problems involving the fractional Laplacian, see the 
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aforementioned recent paper [15] of de Pablo and Sanchez and also [14]. Unlike 
as in [27], we are not able to implement a moving sphere argument directly in the 
present setting, so instead - as in [15] - we first transform (1.1) to a local problem 
by considering the Caffarelli-Silvestre extension of a solution u on see [11] 

and also [7, 17]. This extension satisfies w = u on and solves in some weak sense 
(see Section 2 for details) the boundary value problem 



(1.16) 



div(t 1_2a Vti>) = inR^ +1 , 
w = onR JV \Q, 



a-2 S dw 



-c N)S lim t s -T- = f{x,w) on tt, 



with the positive normalization constant cn, s = ^J/iJ+il, (note that this constant 

2sl ( J ) 

is different from the one noted e.g. in [7, Remark 3.11] due to our normalization 
of (— A) s ). Here and in the following we write z = (x,t) € R+ +1 with x € M. N 
and t > 0, and we identify R N with <9M+ +1 , so that Q is contained in <91R+ +1 . We 
will then apply the moving sphere argument to the local problem (1.16) in place 
of (1.1). We note that the Caffarelli-Silvestre extension of a solution of (1.1) has 
received considerable attention in recent years due to its usefulness in the context 
of many different problems, see e.g. [9,10,13,16,30]. 

We should mention that - in contrast to the nonexistence results for (1.14) based 
on the Pohozaev type identity - our approach does not extend to sign changing 
solutions. The existence resp. nonexistence of sign changing solutions of (1.1) 
under supercriticality and star-shapedness assumptions therefore remains an open 
problem. 

Finally, we would like to compare (1.1) with the related problem 



(1.17) 



A s u = /(x, u) in O, 
u = on dQ. 



Here A stands for the negative Laplacian as a self adjoined operator in L 2 (VL) with 
domain 

as a distribution}, 

and A s is the corresponding power in spectral theoretic sense. Although problems 
(1.1) and (1.17) look similar, there are crucial differences as discussed e.g. in [17]. 
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In particular, solutions of (1.17) have in general much better boundary regularity 
than solutions of (1.1), and this can also be seen when comparing the corresponding 
extended problems. We point out that in [5, 8, 12, 15, 32] a variant of the Caffarelli- 
Silvestre extension for solutions of (1.17) was considered which preserves the regu- 
larity properties up to the boundary. Moreover, nonexistence results for (1.17) have 
recently been proved in [5,32] via a Pohozaev type integral identity for the extended 
problem. As we pointed out before, such an approach is not available for (1.1) resp. 
(1.16) due to the lack of boundary regularity of solutions. 

The paper is organized as follows. In Section 2 we discuss a suitable weak notion 
of solution of (1.16), and we study how problems (1.1) and (1.16) transform under 
a Kelvin type transform. We also formulate two versions of boundary maximum 
principles related to a linearized version of problem (1.16). Since this section deals 
with all technical aspects of the problem, the remaining parts of the proofs of our 
main results are relatively short, and they are contained in Section 3. 

Acknowledgments: This work is supported by the Alexander von Humboldt foun- 
dation. The authors would like to thank the referee for his/her valuable remarks. 

2 Some preliminaries 

Throughout the paper, we consider s £ (0,1) and assume that N > 2s. In this sec- 
tion we collect preliminary tools related to (1.1) and the reformulated version (1.16). 
We also need to introduce some definitions concerning notions of weak solutions. If 
C R N is an open set and / <G Lj oc (Q), we say that u G V s ' 2 (R N ) is a distributional 
solution of (— A) s u = f in Q if 



where (•, -)x>*,2 is defined in (1.4). Note that by considering u G T> S > 2 (R N ) we do not 
prescribe u on R N \ Q here. We start with the following result. 

Lemma 2.1 Let CI be a bounded open set. Then there exists a constant C = 
C(N, s, Q) > such that for all (p G C 2 (O), x E M. N and e E (0, 1) we have 



(2.1) 





(2.2) 
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Proof. For x G R N and e > 0, integration by parts yields 

f --0L iv _ r /• Vv (x + ((v - x)) ■ - — ^—mw.Aydt 

J\x-y\>e\x-y\ + JO J\x-y\>e \X-y\ N+2s 




N + 2s-2\J J\ x - yl=£ 

+ 



Vip(x + t(y - x)) .(y- x )\x- y\~ N - 2s+l do(y)dt 
!\ f ^(x + t(y-x))\x-y\- N - 2s+2 dydt), 

JO J\x-y\>e J 

whereas 

C I V V (x + t(y - x)) -(y-x)\x- y\- N - 2s+1 da(y)dt 

Jo J\x-y\=e 

= e 1_2s / / Vip(x + tea) ■ adadt 

Jo Js N - 1 

= e 1 ~ 2s / Vip(x) ■ adadt + e 2 ~ 2s \ t / D 2 ip(x + etra)[a] ■ adadrdt 

Jo Js 1 *- 1 Jo Jo Js 1 *- 1 

and, by oddness, 

N 



! Vtp(x)-ada = V ^(x) f a' l da = 0. 
J s n-i ^ ' jr[dx^ J J sN -i 



Consequently, 



f ^ ^ dy = \\ ! ^{x + t{y-x))\x-y\- N - 2s+2 dydt 

J\x-y\>e \x-y\ N+2s N + 2s-2j J lx _ y{>£ ^ yy }n m 

£2(1--) f 1 f 1 f 

(2.3) + — — -/ t / D 2 ip(x + etra)[a] • adadrdt, 



N + 2s-2 

while 



lo Jo Js N - 1 



(2.4) /*/ / D 2 ip(x + etra)[a] ■ adadrdt < Ci|M| C 2(]rjv) 

with a constant C\ > depending only on TV and s. We now fix R > such that 
C B(0,R), and we first consider x G \ 5(0, 4R). Then |x - y| > R + M for 
i/efi and therefore 



(2.5) 



/" ip(x) - <p(y) 

J\x-y\>s \x-y\ N + 2 ° V 



~ J lyl < R{R+ \A)N + 2s y ~ 2 l + \ x \N+2s 



with a constant C2 > depending only on R, N and s. Next we consider x G 
B(0,AR) and note that, for every t G (0, 1), 

i? + Ixl 5R 
F — y| < - < — 11 \x + t(y — x)\ < R, 

and 

Ay>(x + % - x)) = K\x + t(y-x)\>R. 
Hence for x G B(0,4R) we have 

/ * / A^(x+t(y - x))\x - y\~ N ~ 2s+2 dydt 

JO J\x-y\>e 

< IMIc 2 ^) / t / l x - y\~ N ~ 2s+2 dydt 

JO J\x+t(v-x)\<R 



ft I 

JO J\x-y\< 



< \\<p\\c*cr») 1*1 \x- y\- N - 2s+2 dydt 



TiB. 



r 1 /*¥ 

< llvllc^CR^)!^" 1 ! y * y r x - 2s drdt 

(2.6) = Mc^s"- 1 ]^^ J\- 1+2s dt = c,\w\\ c , 



'0 

with a constant C3 > depending only on R, N and s. Combining (2.3), (2.4), (2.5) 
and (2.6), we find that there exists a constant C > depending only on R', N and 
s such that (2.2) holds, as claimed. □ 

Next, we consider the conformal diffeomorphism 

(2.7) k : M. N \ {0} —7- W N \ {0}, K (x) = ^. 

It is easy to see that 

(2.8) \k(x) -n(y)\ = , 7 , for every x, y G l w \ {0}, 

Fi |y 1 

and that the Jacobian determinant of k satisfies 

I det J K (x)\ = \x\- 2N . 
In the following, for a measurable function u on M w , we a.e. define Ku on 1^ by 

Ku{x) = \x\ 2s - n u{k{x)). 
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The map K is usually called Kelvin transform, and it is a well known tool in potential 
theory and partial differential equations. It has also been studied in detail in a 
probabilistic framework for stable processes, see [4] and the references therein. Here 
we need the following property of K. 



Lemma 2.2 The map K defines an isometry on V S,2 (R N ), i.e. for every u, v € 
V S > 2 (R N ) we have Ku,Kv € V S ' 2 (R N ) and 

(2.9) (u,v)j)s,2 = (Ku, Kv )x>s,2. 

Proof. Since C™(R N \{0}) is dense in C™(R N ) with respect to the P s ' 2 (R JV )-norm 
as a consequence of our general assumption N > 2s (see [25, p. 397]), it suffices to 
show (2.9) for u, v € C%°(R N \ {0}). By changing variables and using (2.8), we have 



{U,V) V S,2 = I 

Jm 



-dxdy 



(u(x) - u(y))(v(x) - v(y)) 

■N 

L™ lr- !j\ S+ ^\.r\- s --*\!j\- x -^ 



R2N \x - y\ N + 2 ° 



{u(k(x)) - u(K(y)))(v(n(x)) - v(it(y))) , . 2A -, , v . 



(u(k(x)) - u(K(y)))(v(K(x)) - v{n{y))) , ,_ N+2s , ,_ N+2s 
I R 2N \x - y\^+ is 

Observe that 

(«(«(*)) -u( K (y)))(v(K(x)) -v(K(y)))\x\- N+29 \y\- N+28 

= (Ku{x) - Ku{y)){Kv{x) - Kv{y)) + Ku{x)v{n{x))[\y\ 2s - N - \x\ 2s ~ N ] 
+Ku(y)v(K(y))[\x\ 2s - N -\y\ 2s ~ N }. 

We therefore have 

(u(x) - u(y))(v(x) - v(y)) _ f (Ku{x) - Ku(y)){Kv{x) - Kv{y)) 



r ( u {x) - u{ y )){v{x) - v (y)) dxdy _ f 

7r2Jv I a; - y\ N + 2s J r2 n 



\x-y\ N + 2s 



dxdy 



nl . f f Ku(x)v{n(x))[\y\ 2s - N - \x\ 2s ~ N ] , , 
+21im / / , v., — -dydx. 

^oJrn J\ x - y \ >£ \x-y\ iy+2s 

It thus remains to prove that 

, . , f f Ku{x)v{K{x))[\y\ 2s ~ N - \x\ 2s ~ N ] , , 

(2.10 lim / / , L^o U -dydx = 0. 

V ' ^j R ,j M>£ \x-y\ N + 2 * 
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To show this, we consider / G C£°(R \{0}) defined by f(x) = Ku{x)v{k{x)). Since 

Ln [ , f \x X -v\"+** dydX < °°' Ln [ , fi-^U^ ^ < °°' 
./R^ J|x-j/|>e l x !/| A" ./|x-j/|>e \ x U\ 

we have by Fubini's theorem 

Note that x h-> |xj 2s_Ar G By Lemma 2.1, we have 



x~y\>s V\ 



^TTW^ for all £G (0,1), 



and therefore the dominated convergence theorem implies that 

lim f f ^f^-jf^ dydx = f \x\ 2s ~ N (- A) s f (x)dx. 



£ ^°Jr» J\x-y\>e \x-y\ N+2s 

Since x i-> \x\ 2s ~ N is the Riesz potential of order 2s, we have (up to a constant) 

/ \x\ 2s - N (-AYf(x)dx = ((-A) s \x\ 2s - N ,f) = (S,f) = 
Jr n 

in distributional sense, because / is supported away from the origin and 5 is the 
Dirac mass at the origin. Hence we have proved (2.10) and the lemma then follows. 
□ 

As a consequence, we get the following result, which is closely related to [4, 
Theorem 2]. We note that, unlike in the present paper, probabilistic techniques are 
used in [4]. 

Corollary 2.3 Let tt C R N be an open set and 

n:=/e(fi\{0}) cK tf \{0}. 

Let f G L} (Q), and let u G D S,2 (W N ) solve (— A) s u = f inQ in distributional sense. 
Then u = Ku is contained in T> S ' 2 (K N ) and solves (—A) s u = f in distributional 
sense in tl, where f G Lj oc (tl) is given by f(x) = \x\~( N+2s ^ f( j^z)- 
Moreover, if u G D s ' 2 (0), then u G £> s ' 2 (ft). 
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Proof. Suppose first that u G V s > 2 (tt). Since, as noted before, C£°(fi\{0}) is dense 
in V S,2 (Q), there exists a sequence (ip n )n in C^°(^l \ {0}) with \\u — ip n \\v B < 2 — > 
as n — > oo. By (2.2), we then also have \\Ku — Kip n \\ V s,2 — > as n — >• oo. Since 
iTVn G £> s ' 2 (^) for all n, this implies K« G £> s > 2 (ft). 

Next we assume that u G X' s ' 2 (IR iV ) solves (— A) s u = / in in distributional sense. 
Applying the argument above to Q = R N yields u G V S > 2 (R N \ {0}) C V S > 2 (R N ). 
Moreover, for given (p G C£°(fi), we may now write <p = Kip with 99 G C£°(fi). By 
Lemma 2.2, we then have 

(u,ip) V s,2 = (u,ip) V s,2 = / fipdx= / (/ o o k) I det J K \ dx 

Jn Jn 

= lf(K(x))(p(K,(x))\x\~ 2N dx= Ifipdx. 
Jn Jn 

This shows the claim. □ 

Next, we introduce some notations related to the reformulated problem (1.16). 
As before, we write z = (x, t) G R+ +1 with x G R N and t G (0, 00). Let D 1 ' 2 (M+ +1 ; i 1_2s ) 
denote the space of all functions w G if/ 0C (R+ +1 ) such that 

i 1_2s |V«;| 2 d2 < 00. 



Formally introducing the operator L s := div(t 1 2s V) on R^ +1 , we say that a func- 
tion w G D 1 ' 2 (R I l +1 ;t 1 ~ 2s ) is weakly L s -harmonic if 



t^ 2s VwV^ dz = for all p G CfV f 



By standard elliptic regularity, every weakly L s -harmonic function w G D 1,2 (R^ +1 ;t 1 ~ 2s ) 
belongs to C°°(R+ +1 ) and satisfies div(i 1_2s Vu> ) = pointwise in R^ +1 . Moreover, 
w does not attain an interior maximum or minimum point in R^ +1 unless w is 
constant. Note also that we have a well defined continuous trace map 

D^iR^-t 1 - 23 ) -^V S ' 2 (R N ) 



(see e.g. [5]), and for the sake of simplicity we denote the trace of a function in 



D 1 > 2 (R^ +1 ; t 1 - 28 ) with the same letter as the function itself. l£ip,ip G D 1 ' 2 (R^ +1 ;t 1_2a ) 
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and <p is weakly L s -harmonic, we have the identity 

(<p(x) - (p(y))(ip{x) -ip(y)) 



(2.11) c Ns [ t 1 - 2s VipVipdz= [ 



\ x _ y \N+2s 



dxdy 



with cjsf^s as in (1.16). Now, for an open set 17 C R , we denote by D(Q,s) the 
closed subspace of functions in D 1,2 (R+ +1 ; t 1_2s ) such that their trace on R^ is 
contained in D s ' 2 '(£1). It is easy to see that every function u G T> S,2 (Q) has a unique 
weakly harmonic extension H(u) G D(£l,s) which can be found by minimizing the 
functional 

! t l - 2s \Vw\ 2 dz 



among all functions w G D(Q,s) satisfying w = u on R . Using this fact in the 
special case O = R^ (in which D(R N ,s) = D 1 ' 2 (R^ +1 ;t 1 ~ 2s )) together with (2.11), 
we find that 

(2.12) / ^-^ dxdyKc^f t^V^dz 
J R 2N \x-y\ N+2s J R N+1 

for all if G Z) 1 ' 2 (M+ +1 ;t 1 ~ 2s ). Moreover, since V s ' 2 (R N ) is continuously embedded 
in L N - 2s (R ), there exists a constant (7 > such that 

(2.13) |M| 2 2N < C [ { ^ X) ~^}f dxdy for all <p G V S > 2 (R N ). 
Another fact we need is the following: 

Lemma 2.4 Let 0, C R N be a bounded open set, and let u G D 1 ' 2 (R+ +1 ; t 1_2s ) be 
such that its trace - also denoted by u - is continuous in 12 and satisfies u = on 
R N \n. Thenue D(n,s). 



Proof. Consider G G C*°°(R) such that 

G(r) = if|r|<l, G{r) = r if |r| > 2 and \G'(r)\ < 3 if 1 < |r| < 2. 

Then the functions ii ra defined by u n (t,x) = -G(nu(t,x)) are clearly contained in 
D 1 ' 2 (R+ +1 ;t 1 - 2s ) for n G N. Passing to traces, we therefore have u n G P 5 ' 2 ^). 
Note that by the dominated convergence theorem we have u n — > u in D 1,2 (R+ +1 ; t 1_2s 
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In addition, since the support of the trace of u n in M N , is contained in the compact 
subset of Q , 

x G SI : > — 

it follows that u n G £> s ' 2 (0) by the density result in [22, Theorem 1.4.2.2]. To 
conclude we observe that u n — >■ u in D s ' 2 (fi) and this holds true thanks to the 
continuity of the trace operator D 1 ' 2 (M^ +1 ; t 1_2a ) ->• V s ' 2 (R N ). □ 

We remark that the continuity assumption in Lemma 2.4 is not needed if SI has 
a continuous boundary, see [22, Theorem 1.4.2.2]. 

Next, let q s := n+ 2 s ^ e * ne conjugate of t^t^- If / € L qs (S}) is given and u G 
P S ' 2 (R JV ) satisfies (— A) s u = f in in distributional sense, then, as a consequence 

2iV 

of the embedding V s ' 2 (SI) <^-» L N - 2s (SI), it also satisfies this equation in weak sense, 
i.e. 

(u,^) V s, 2 = [ fipdx for all V G P s ' 2 (^). 

Moreover, by (2.11), the weakly L^-harmonic extension w = H(u) € D(Sl,s) of u 
then satisfies 

(2.14) c N , s / t l ~ 28 VwVipdz= fi>dx for all ^ € Z>(fi, s). 

We may summarize the discussion in the following statement. 



Lemma 2.5 Let SI C R 6e an open set and f G L qa (Sl). A function w G 
D 1,2 (R+ +1 ; t 1_2s ) satisfies (2.14) if an d only if w is weakly L s -harmonic and its 
trace - also denoted by w G V S ' 2 (M. N ) - solves (— A) s w = f in SI in distributional 
sense. 

If this holds, we say that w weakly solves the problem 



(2.15) 



div^-^Vw) = mR^+\ 



-CTVslimt s wt = f onS}. 



Next, we examine how problems of type (2.15) transform under generalized 
Kelvin inversions. 
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Proposition 2.6 Let w G D 1 ' 2 ^" 1 " 1 ;* 1-2 *), /et 11 C R w an open se£ and Zei 
/ G L qs (£l). Moreover, for fixed p > 0, consider 

2 



and let w p : E+ +1 -> E ; / p : Q p -> E 6e defined by 

( P\ N ~ 2S {P 2 A ( p\ N+2s fp 2 x 

wp{z):= \\A) W \W) and fp{x) = \¥\) f \W 

Then we have: 
(%) w p G D 1 ' 2 (E^ +1 ;i 1 - 2s ), and f p G L qs (Q p ) . 
(ii) If w weakly solves the problem 

div(t 1_2s Vii;) = 



(2.16) 

then w p weakly solves the problem 



1- jl-2s^ W J! o 

c N , s hmt — — = / on J 2, 



div(t 1 - 2s Vn; p ) = in ™ N+1 
(2.17) <( ^ 

-CAr^limt 1 2s ^-^ = f p on f2„. 



+ ' 



Proof. Let w G L> 1 ' 2 (E^ +1 ; t^ 2s ) and / G L««(n). Note that n; p (z) = p 2s ~ N w 1 {j I ) 
and / p (a;) = p~ (iV+2s) u>i(^) for every p > 0, z G E^ +1 and x G E w \ {0}. Hence 
it suffices to prove the claims in the case p = 1, and we put w = w\, f = fi and 
SI = Oi . Recalling the properties of the map k defined in (2.7), we then find 

[_\f\«'dx= [\x\- 2N \f(j^)\1°dx= [_\J K \\f0K\<*dx= f \f\*'dx. 

Jn Jn \ x \ Jn Jn 

To simplify the notations, we set 

r : ij^ \ {0} -+ ij^ \ {0}, t{z) = |^r, 

so that the restriction of r to E^ \ {0} coincides with k. We note that the Jacobian 
J T of r satisfies 

J?(z)J T (z) = \z\- 4 I, 
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where / denotes the (n + 1) x (n + l)-identity matrix, and det J T (z) = \z\ for 
every z G 



Next, we write w = goT with g(z) = \z\ N 2s w(z). Moreover, we let <p G C£°(R+ +1 \ 
{0}) be arbitrary, and define <p G C C °°(IR^ +1 \ {0}) by <p = h o r with h(z) = 
p(z). Considering first the special case where w G C C °°(M^ +1 \ {0}), we then 



N-2s 



calculate 



/ t^VwVpdz = I t l - 2s [J T {z)Vg{r{z))\ ■ [J T (z)Vh(r(z))}dz 

= f t^lzl^VgMz^VhMz^dz 
Jr^ +1 

= J rN+1 kr 2JV ~ 2 (^) 1 ' 2 %! 2 ^V,(r(,))V/ i (r(,))^ 

= I | det J T (z)|(^ I ) 1 " 2 VW| 2(2s - iV) V 5 (r(^))V/ l (r(^))^ 

« I I 

= f t 1 - 2s \z\^ 2s - N ^g(z)Vh(z)dz. 

Noting that 

Vg(z) = (N- 2s)\z\ N ~ 2s ^ 2 zw{z) + \z\ N ' 2s Vw{z) 

and 

Vh{z) = {N- 2s)\z\ N - 2s ~ 2 z^{z) + \z\ N - 2s Vip(z), 
we then conclude that 

f t 1 ~ 2s VwV$dz= f t 1 - 2s VwVcpdz + h + I 2 + h 

Jr^ 1 ' Jr^ 1 

with 

h = (N- 2s) 2 [ t^h^wiz^iz) dz, 

1 2 = (N-2s) [ t 1 ' 2s \z\~ 2 w(z)zVip(z)dz, 

JR^ +1 

1 3 = (N-2s) [ t 1 - 2s \z\- 2 ^(z)zVw(z)dz. 

Jr^ 1 
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Since div z [t 1-2a |z| -2 z] = (N - 2s)t 1 - 2s \z\- 2 , it follows that 

h + h + h = (N-2s) [ div z ( t 1 ' 23 \z\~ 2 zw(z)ip(z)) dz = 
and therefore 

(2.18) f t l - 2s VwV(pdz = [ t l - 2s VwV^dz. 



By [19], we have that C~(R^ +1 \ {0}) is dense in D 1 ' 2 (R^ +1 ; i 1_2s ) thus we deduce 
that (2.18) also holds for arbitrary w,<p G D 1,2 (R+ +1 ;i 1-2s ), while w,<p are also 
contained in Z} 1,2 (R+ +1 ; t 1_2s ). In particular, (i) is proved. 

Moreover, (2.18) implies that w; is weakly L s -harmonic if u; is weakly L s -harmonic. 
In addition, considering the traces of w and w respectively, Corollary 2.3 implies 
that (— A) s w = f in distributional sense in if (— A) s w = f in distributional sense 
in Q. Hence (ii) follows from Lemma 2.5. □ 

We will need the following version of a strong maximum principle which is es- 
sentially a reformulation of [7, Proposition 4.11]. 

Lemma 2.7 Let E be an open subset of R N , and let w G D 1,2 (R+ +1 ; t 1-2s ) be a 
weak solution of 

div(* 1_2a Vu;) = inR^ +1 , 

-c Ns hm t -wr = 9 on E 
t^o+ at 

for some g G L Qs (E) n C(E). Suppose furthermore that w is continuous and non- 
negative on E x [0, r] for some r > 0, and that 

(2.19) g(x) > for every x £ E with w(x) = 0. 

If w ^ in E, then w is strictly positive in E and therefore inf w > for any 
compact set K C E. 

Proof. If w on E 1 , then iw > in E x (0, r), since w is L s -harmonic and 
nonnegative in this set. Suppose by contradiction that w(xo) = for some xo G E. 
Then g(x ) < by [7, Proposition 4.11], which contradicts (2.19). □ 
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We will also need the following "small volume" maximum principle: 



Lemma 2.8 Let 7 > 0. Then there exists 5 = 5(N,s,j) > with the following 
property. If 



(i) F C is an open subset with dF n R N + 0, 

(ii) E is a bounded open subset ofM. N with E C dF, 
(Hi) c G L°°(E) is given with \\c\\ioo^ < 7, 

(iv) w G D 1 ' 2 (R^ +1 ;t 1_2s ) is a weak solution of 

div(^ 2s Vu>) < 



(2.20) 



in 



1 o 9u) 
-cat jS lim t "ttt — c(z)iu 
t->-o+ at 



1 

+ ' 
on E, 



i.e., 

(2.21) c N)S / t 1-2s Vu;V<£>d,z > / c(x)u/<£> 

J F J E 

for all nonnegative 92 G D(E, s), 

(v) w is continuous on F and satisfies w > on dF \ E, 

(vi) \{x G E : w < 0}| < 5, 
then w > in F. 



dx 



Proof. We consider the function 



v : 



pAT+l 



v(x) = < 



max(— w(x), 0), x £ F, 







x G 



!,N+1 



\F. 



It can be deduced from assumptions (i) and (ii) that the relative boundary of F in 



is contained in dF \ E, so that v is continuous on by assumption (v). 

Moreover, v = on M N \ E. As a consequence, v G Hf oc (R^ +l ) by [6, Theorem 9.17 
and Remark 19], and 

f t^lVvl 2 dz < [ t 1 ' 23 \V w\ 2 dz < 00. 
Jr n + 1 Jr n+1 
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Hence v G D 1 ' 2 (R^ +1 ; i 1_2s ), and Lemma 2.4 implies that v G D(E,s). We also 
note that combining (2.12) and (2.13) yields a constant C = C(N,s) > such that 

^«/(w-m (»iv\ 5; C / 2s |V"u| 2 <iz. 



M| L 2JV/(AT-2 s )( K iV) 



Applying (2.21) to u, we then obtain 

c Ns / i 1-2s |Vi;| 2 dz = -c N s / £ 1_2s Vw ■ Vvdz < - / c(x)wvdx 

= j^c{x)v 2 dx < \\c\\ L oo( E )\{x G E : w < 0}\ N/2s \\v\\ 2 l2N/(n _ 2s)(:rN ^ 
< 1 5 N ' 2s C [ t^Vvfdz. 

Hence, if S < (j^fj ^ , then v = in and therefore w > in F, as claimed. 

□ 

3 Proof of the main results 

In this section we complete the proof of our main results. We begin with the 
Proof of Theorem 1.1: 

We suppose by contradiction that there exists a nontrivial solution u G C(R N \ 
{0}) n V S ' 2 (Q) of (1.1), and we let w G Z> 1,2 (R^ +1 ; t 1_2s ) denote the corresponding 
L s -harmonic extension of u which weakly solves the problem 

div(* 1_2fl Vu;) = in 
-CN, s h™t 1 ~ 2s wt = f(x,w) onfl', 

for every open subset Q' C Q which is relatively compact in M^\{0}. Here, as before, 
we also write w in place of u for the trace on M N . We clearly have w G C(1R^ +1 \{0}). 
Let R := sup{|x| : x G ^2} > 0. For p G (0, R), we consider the Kelvin transform 
w p of w as defined in Proposition 2.6. We also put 

2 

F p := {z G : |z| > p}, E p := {x G : |x| > p} and E p := {j-^ : x G E p }. 
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By definition of R and since 0, is star-shaped with respect to the origin, E p and 
E p are nonempty open subsets of SI which are relatively compact in R N \ {0} for 
p € (0,-R), so that the restrictions of the map x i-> f(x,w(x)) to E p and E p are 
bounded and continuous. By Proposition 2.6, the difference function v p = w p — w € 
D 1 ' 2 (R^ +1 ;t 1_2i ) weakly solves the problem 



div(t L - 2s Vv p ) = 

A l-2sr 



m 



-cat, s lim t s [w p ]t = g p on £ 



+ ' 



where g p is the bounded and continuous function on E p given by 

/ n N+2s / 2 / \ 2s-N \ 

Moreover, by the supercriticality assumption (1.6) we have 

9 P {x) > f(x,w p (x)) - f(x,w(x)) = c p (x)v p (x) for x in E p 

with 

' f(x,w p (x)) - f(x,w(x)) 



c p : E p ^ R, c p (x) = < 



w p (x) — w{x) 







if w(x) + w p (x), 
if w(x) = w p (x). 



We also note that, since / is assumed to be locally Lipschitz in its second variable, 
we have c p € L°°(E p ) for < p < R, Moreover, for r G (0, R) we have 



(3.1) 

We now define 



7 T := sup ||c|[ L oo ( £ p) 

pe[r,R) 



< oo 



p* := inf{p € (0, R) : v p > in F p for p € [p, i?)}. 

Since \E p n{w p < 0}| is small provided p is sufficiently close to R, Lemma 2.8 implies 
that p* < R. We claim that p* = 0. Indeed, suppose by contradiction that p* > 0. 
By continuity, we then have v Pt > in F Pt . Moreover, v Pt ^ in E Pt since 



v Pf (x) > for every x G dil with \x\ > p*. 
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By Lemma 2.7, we obtain v Pt > in E Pt . We now fix r G (0, p*) and choose 5 > 
as in Lemma 2.8 according to 7 = j T as denned in (3.1). Moreover, we choose a 
compact set K C E pt such that \E Pt \K\ < 6. Then inf w pt > 0, and by continuity 
we also have 

K C E p , \E p \K\<5 and ^ w p > 

for p G (r, p*) sufficiently close to p*. Therefore Lemma 2.8 implies that v p > in 
F p for p G (r, p* ) sufficiently close to p* . This contradicts the definition of p* . We 
conclude that p* = 0, as claimed. As a consequence, for every and i/0we 

have 

/ \ N~2s / 2 \ 

(3.2) u; > u/(x) for all p € (0, |x|). 

Furthermore, since w G D s ' (f2) C L N - 2s (R ) we have 

/" Z" 00 2iV /" 2JV 

/ / w N ~ 2s (ra)drda = / w N ~ 2s dx < 00 
and therefore, by Fubini's theorem, 

/•°° 2iv , 

(3.3) / w N ~ 2s (rao)dr < 00 for a.e. <7o G 
Jo 



-iN-l 



We now pick do G 1 and tq > such that ro<To G SI and (3.3) holds for a®. By 
(3.2) we then have 

\ N-2s / 2 \ 
Z. ) w ( ) > u,(r CT ) > for p G (0, r ) 
?"0/ V r o / 

and consequently 

w(ro-o) > Cr^ s ~ N ^ 2 for r G (0, r ) with a constant C > 0. 
This implies 

/" r » 2iV 

/ w n-2s (roo)dr = 00 
Jo 

contrary to (3.3). The contradiction shows that there does not exist a nontrivial 
solution u G C(R N \ {0}) DV S ' 2 (Q) of (1.1) under the assumptions of Theorem 1.1, 
as claimed. □ 
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Proof of Corollary 1.2: 

Problem (1.9) is a special case of (1.1) with f(x,u) = u p — V(x)u, and for this 
nonlinearity we calculate 

H f ( X ,u) ={p- ^fjy + j^y s { sV{x) + \ x ■ w(x) ) 

so that (1.7) is satisfied by the assumptions on p and V. Moreover, any nontrivial, 
nonnegative solution of (1.9) is strictly positive in Q\{0}, which follows by applying 
Lemma 2.7 to the L s -harmonic extension of u and the sets E e := {x G f2 : \x\ > e} 
for e > small. Hence nontrivial, nonnegative solutions of (1.9) do not exist by 
Theorem 1.1. □ 

Proof of Corollary 1.3: 

Problem (1.10) is a special case of (1.9) with V(x) = 7|x|~ 2s , so the result follows 
from Corollary 1.2. □ 

Proof of Corollary 1.4: 

Problem (1.11) is a special case of (1.1) with f(x,u) = \x\~ a u p , and for this nonlin- 
earity we calculate 

N + 2s-2a^ 



H f (x,u) = (p- N _ 2g )\x\ u*. 



Hence (1.7) is satisfied by the assumptions on p and a. Moreover, by the same 
argument as in the proof of Corollary 1.2 above, nontrivial and nonnegative solu- 
tions of (1.11) must be strictly positive in Q \ {0} and therefore can not exist by 
Theorem 1.1. □ 

We finally give the proof of our nonexistence result in (unbounded) domains being 
star-shaped at infinity. 

Proof of Theorem 1.5: 

The definition of star-shapedness at infinity implies that, after a suitable translation, 
the image Q := k(Q) of the domain $7 under the map k defined in (2.7) is star-shaped 
with respect to € d£l. Moreover, if u G U s > 2 (Q) C\C(R N ) is a nonnegative solution 
of (1.12), then Corollary 2.3 implies that u = Ku G V s < 2 (n) n C(R N \ {0}) solves 
(— A) s u = |x| _CT |tt| p in Q with a = N + 2s — p(N — 2s). Since the assumption 
P < yields p > Af ^ 2 !^ j 2CT , Corollary 1.4 implies that u = and hence also 

u = 0, as claimed. □ 
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